Architecture for Efficient Implementation of the YK Lossless Data Compression Algorithm

The YK (Yang-Kieffer) algorithm [1,2] is a sequential lossless data compression built upon an efficient greedy grammar transform. This grammar transform constructs a sequence of irreducible context-free grammars from which the original data sequence can be recovered incrementally. The transform sequentially parses the original data into nonoverlapping variable-length phrases, and updates the grammar accordingly. Compression of data is achieved by using an arithmetic coder for indirectly encoding the context-free grammars that represent the original data sequence. The YK algorithm jointly optimizes, in some sense, string matching and arithmetic coding capabilities, to achieve excellent compression performance on a large class of data. Experimental results show that the YK algorithm significantly outperforms other commonly used lossless data compression algorithms such as Lempel-Ziv types of codes. Moreover, the YK algorithm is highly effective on small files such as internet datagrams, which are usually only a few hundred bytes long. However, practical implementation of the YK algorithm is non-trivial and therefore is of considerable research interest. Our paper proposes a novel architecture for the implementation of the YK algorithm that is motivated by hardware issues. However, the proposed implementation lends itself to an efficient software implementation on embedded processors as well. The overall architecture consists of three modules corresponding to the three primary steps of the YK algorithm  parsing, grammar transform and arithmetic coding. A parse trie data structure is used for efficient parsing. A cross-referencing pair of data structures is used to represent the grammar and lists. This allows a strict control over the memory requirement of the algorithm, and considerably simplifies the grammar and list update procedures. The arithmetic coding step is simplified at negligible cost to compression efficiency, leading to natural separation of grammar transform and arithmetic coding modules. The inverse grammar transform at the decoder is simplified by exploiting partially decoded data at each iteration step. The overall architecture is illustrated below: